In this paper we present a new method for deformable NeRF that can directly use optical flow as supervision. We overcome the major challenge with respect to the computationally inefficiency of enforcing the flow constraints to the backward deformation field, used by deformable NeRFs. Specifically, we show that inverting the backward deformation function is actually not needed for computing scene flows between frames. This insight dramatically simplifies the problem, as one is no longer constrained to deformation functions that can be analytically inverted. Instead, thanks to the weak assumptions required by our derivation based on the inverse function theorem, our approach can be extended to a broad class of commonly used backward deformation field. We present results on monocular novel view synthesis with rapid object motion, and demonstrate significant improvements over baselines without flow supervision.
more »
« less
Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to non-federal websites. Their policies may differ from this site.
-
-
Multi-view triangulation is the gold standard for 3D reconstruction from 2D correspondences given known calibration and sufficient views. However in practice, expensive multi-view setups – involving tens sometimes hundreds of cameras – are required in order to obtain the high fidelity 3D reconstructions necessary for many modern applications. In this paper we present a novel approach that leverages recent advances in 2D-3D lifting using neural shape priors while also enforcing multi-view equivariance. We show how our method can achieve comparable fidelity to expensive calibrated multi-view rigs using a limited (2-3) number of uncalibrated camera views.more » « less
-
more » « less
Abstract Electrochemical random‐access memory (ECRAM) is a recently developed and highly promising analog resistive memory element for in‐memory computing. One longstanding challenge of ECRAM is attaining retention time beyond a few hours. This short retention has precluded ECRAM from being considered for inference classification in deep neural networks, which is likely the largest opportunity for in‐memory computing. In this work, an ECRAM cell with orders of magnitude longer retention than previously achieved is developed, and which is anticipated to exceed ten years at 85 °C. This study hypothesizes that the origin of this exceptional retention is phase separation, which enables the formation of multiple effectively equilibrium resistance states. This work highlights the promises and opportunities to use phase separation to yield ECRAM cells with exceptionally long, and potentially permanent, retention times.
